On the Determination of the Number of Signals and Its Performance Analysis in Presence of White Noise

In signal processing, high resolution signal parameter estimation is a significant problem. In particular the estimation of the direction of the narrow band signals emitted by multiple sources received wide applications recently in signal processing literature. Quite a number of papers appeared in the last twenty five years regarding the estimation of the parameters of the direction of arrival of signals, but not that much attention has been given in estimating the number of signals. In this paper we develop a method using penalty function technique. But instead of using any fixed penalty function like AIC or MDL, a class of penalty functions satisfying some special properties have been used. We prove that any penalty function from that particular class will produce consistent estimates under the assumptions that the error random variables are independent and identically distributed with mean zero and finite variance. We also obtain the probabilities of wrong detection for any particular penalty function and estimate it using the matrix perturbation technique. It gives some idea to choose the proper penalty function for any particular model. Simulations are performed to verify the usefulness of the analysis and to compare our method with the existing ones. AMS Subject Classifications: 62H12, 62H15, 62F12